Tangle (Export the files)
--- General
- [ ] Add a documentation system (finding a way for html and pdf export ?)
- [ ] Document every function correctly (docstrings)
- [ ] Add some examples
- [ ] Error/Warning managment
- [ ] Check for other OS compatibility
- [ ] TODO tags (and others)
- [ ] Add more types of data (List…)
0.03a
- External changes
Objects (Schedule, Clock, Drawer, Table, Node) are now called OrgSchedule, OrgClock, OrgDrawer, OrgTable, OrgNode to avoid conflicts
OrgSchedule: Added the support for combo lines (SCHEDULED+DEADLINE+CLOSED)
OrgClock: Now uses the new OrgDate class
DataStructure is now called OrgDataStructure
- Internal changes
Restructuration of the .org tangling-file (Document, Code and Tests sections for each kind of class)
Restructuration of Plugin and Elements system (mainly to keep the indentation)
OrgDate: A new class for managing date and times (and active/inactive time-date formats)
Included the patch from KAIHOLA Antti about a wrong parenting bug
- Documentation
Added some details about functions (a lot of work remains)
- Tests
Added the test from KAIHOLA Antti for the parser
Added the different test tools from KAIGOLA Antti (like test_simple-agenda updated to use unittest)
- Python compliance
Added the minimal setup file of KAIHOLA Antti
0.02b
- External changes
Added the patch from KAIHOLA Antti about time format.
0.02a
- External changes
Added a Clock plugin in order to manage CLOCK: elements
Thank to "siberianlaika" for the idea
Added a TYPE attribute to the different OrgPlugins elements
Maybe we should find a more practical way than this one
Removed some debug "prints"
Indentation problem that causes wrong reparenting
Thanks to Matthew Robison for his patch
Added a append_clean function to Node class
This function is used to add a tree to a node
A call to reparent_cleanlevels is done at the end
Added reparent_cleanlevels to Node class
This function reparent all the elements. Using a
content to parent way of doing. It checks the content
of the first element (call it E), and set the parent of
each E-childs to E. This is really useful when moving
one tree to another place.
Loading the default plugins in the Node class init.
This avoids to do it in the load_from_file (which was a
bad idea).
Added a new example
using the TYPE attribute and append_clean.
0.01j
- External changes
Added load_plugin function to DataStructure
0.01i
- Internal changes
Renamed Plugin class to OrgPlugin
Added close function to plugins
Adding Table cells subdivision (easier editing)
- PyOrgMode.org Structure change
Added test.py in the document
0.01h
- Internal changes
Added Plugin system (simplifying the main loop of DataStructure)
The DataStructure class is now an OrgElement
- External changes
Node,Table,Drawer and Schedule are now plugins.
Their object method is now joined by PluginName.Element
- New elements
Added Table element (as a Plugin)
0.01g
- Changed elements
Node : Added priority management
0.01f
- New elements
Added Schedule element for 'DEADLINE: and 'SCHEDULED:
- Optimizations
Class DataStructure : Trying to simplify the Reg exps
- [X] BISSON Jonathan <bissonjonathan on the googlethingy>
- [X] KAIHOLA Antti <akaihol plus orgmode at ambitone dot com>
# -*- encoding: utf-8 -*-
##############################################################################
#
# PyOrgMode, a python module for treating with orgfiles
# Copyright (C) 2010 Jonathan BISSON (bissonjonathan on the google thing).
# All Rights Reserved
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
##############################################################################from setuptools import setup
setup(
name='PyOrgMode',
version='0.03a',
py_modules=['PyOrgMode'],
)"""
The PyOrgMode class is able to read,modify and create orgfiles. The internal
representation of the file allows the use of orgfiles easily in your projects.
"""
import re
import string
import copy
import time--- Class OrgDate
- [ ] Must support locale (conversion for example)
- [ ] Must support empty initialisation
- [ ] Must use data validation
- [ ] Must support recurrent events (+1w …)
class OrgDate:
"""Functions for date management"""
format = 0
TIMED = 1
DATED = 2
WEEKDAYED = 4
ACTIVE = 8
INACTIVE = 16
def __init__(self,value=None):
"""
Initialisation of an OrgDate element.
"""
if value != None:
self.set_value(value)
def set_value(self,value):
"""
Setting the value of this element (automatic recognition of format)
"""
# Checking whether it is an active date-time or not
if value[0]=="<":
self.format = self.format | self.ACTIVE
value = re.findall("(?:<)(.*)(?:>)",value)[0]
elif value[0]=="[":
self.format = self.format | self.INACTIVE
value = re.findall("(?:\[)(.*)(?:\])",value)[0]
# Checking if it is a date, a date+time or only a time
value_splitted = value.split()
timed = re.compile(".*?:.*?")
dated = re.compile(".*?-.*?-.*?")
if timed.findall(value):
self.format = self.format | self.TIMED
if dated.findall(value):
self.format = self.format | self.DATED
if len(value_splitted) == 3 :
# We have a three parts date so it's dated, timed and weekdayed
self.format = self.format | self.WEEKDAYED
self.value = time.strptime(value_splitted[0]+" "+value_splitted[2],"%Y-%m-%d %H:%M")
elif len(value_splitted) == 2 and (self.format & self.DATED) and not (self.format & self.TIMED):
# We have a two elements date that is dated and not timed. So we must have a dated weekdayed item
self.format = self.format | self.WEEKDAYED
self.value = time.strptime(value_splitted[0],"%Y-%m-%d")
elif self.format & self.TIMED:
# We have only a time
self.value = time.strptime(value,"%H:%M")
elif self.format & self.DATED:
self.value = time.strptime(value,"%Y-%m-%d")
def get_value(self):
"""
Get the timestamp as a text according to the format
"""
if self.format & self.ACTIVE:
pre = "<"
post = ">"
elif self.format & self.INACTIVE:
pre = "["
post = "]"
else:
pre = ""
post = ""
if self.format & self.DATED:
# We have a dated event
dateformat = "%Y-%m-%d"
if self.format & self.WEEKDAYED:
# We have a weekday
dateformat = dateformat + " %a"
if self.format & self.TIMED:
# We have a time also
dateformat = dateformat + " %H:%M"
return pre+time.strftime(dateformat,self.value)+post
elif self.format & self.TIMED:
# We have a time only
timestr = time.strftime("%H:%M",self.value)
if timestr[0] == '0':
return timestr[1:]
return pre+timestr+postimport PyOrgMode
import time
import unittest
class TestClockElement(unittest.TestCase):
def test_duration_format(self):
"""Durations are formatted identically to org-mode"""
for hour in '0', '1', '5', '10', '12', '13', '19', '23':
for minute in '00', '01', '29', '40', '59':
orig_str = '%s:%s' % (hour, minute)
orgdate_element = PyOrgMode.OrgDate(orig_str)
formatted_str = orgdate_element.get_value()
self.assertEqual(formatted_str, orig_str)
if __name__ == '__main__':
unittest.main()--- Class OrgList
- [ ] Must be written
--- Class OrgProtocol
- [ ] Must be written
class OrgPlugin:
"""
Generic class for all plugins
"""
def __init__(self):
""" Generic initialization """
self.treated = True
self.keepindent = True # By default, the plugin system stores the indentation before the treatment
self.keepindent_value = ""
def treat(self,current,line):
""" This is a wrapper function for _treat. Asks the plugin if he can manage this kind of line. Returns True if it can """
self.treated = True
if self.keepindent :
self.keepindent_value = line[0:len(line)-len(line.lstrip(" \t"))] # Keep a trace of the indentation
return self._treat(current,line.lstrip(" \t"))
else:
return self._treat(current,line)
def _treat(self,current,line):
""" This is the function used by the plugin for the management of the line. """
self.treated = False
return current
def _append(self,current,element):
""" Internal function that adds to current. """
if self.keepindent and hasattr(element,"set_indent"):
element.set_indent(self.keepindent_value)
return current.append(element)
def close(self,current):
""" A wrapper function for closing the module. """
self.treated = False
return self._close(current)
def _close(self,current):
""" This is the function used by the plugin to close everything that have been opened. """
self.treated = False
return currentclass OrgElement:
"""
Generic class for all Elements excepted text and unrecognized ones
"""
def __init__(self):
self.content=[]
self.parent=None
self.level=0
self.indent = ""
def append(self,element):
# TODO Check validity
self.content.append(element)
# Check if the element got a parent attribute
# If so, we can have childrens into this element
if hasattr(element,"parent"):
element.parent = self
return element
def set_indent(self,indent):
""" Transfer the indentation from plugin to element. """
self.indent = indent
def output(self):
""" Wrapper for the text output. """
return self.indent+self._output()
def _output(self):
""" This is the function really used by the plugin. """
return ""
def __str__(self):
""" Used to return a text when called. """
return self.output()class OrgClock(OrgPlugin):
"""Plugin for Clock elements"""
def __init__(self):
OrgPlugin.__init__(self)
self.regexp = re.compile("(?:\s*)CLOCK:(?:\s*)((?:<|\[).*(?:>||\]))--((?:<|\[).*(?:>||\])).*=>\s*(.*)")
def _treat(self,current,line):
clocked = self.regexp.findall(line)
if clocked:
self._append(current,self.Element(clocked[0][0], clocked[0][1], clocked[0][2]))
else:
self.treated = False
return current
class Element(OrgElement):
"""Clock is an element taking into account CLOCK elements"""
TYPE = "CLOCK_ELEMENT"
def __init__(self,start="",stop="",duration=""):
OrgElement.__init__(self)
self.start = OrgDate(start)
self.stop = OrgDate(stop)
self.duration = OrgDate(duration)
def _output(self):
"""Outputs the Clock element in text format (e.g CLOCK: [2010-11-20 Sun 19:42]--[2010-11-20 Sun 20:14] => 0:32)"""
return "CLOCK: " + self.start.get_value() + "--"+ self.stop.get_value() + " => "+self.duration.get_value()+"\n"--- Class OrgSchedule
class OrgSchedule(OrgPlugin):
"""Plugin for Schedule elements"""
# TODO: Need to find a better way to do this
def __init__(self):
OrgPlugin.__init__(self)
self.regexp_scheduled = re.compile("SCHEDULED: ((<|\[).*?(>|\]))")
self.regexp_deadline = re.compile("DEADLINE: ((<|\[).*?(>|\]))")
self.regexp_closed = re.compile("CLOSED: ((<|\[).*?(>|\]))")
def _treat(self,current,line):
scheduled = self.regexp_scheduled.findall(line)
deadline = self.regexp_deadline.findall(line)
closed = self.regexp_closed.findall(line)
if scheduled != []:
scheduled = scheduled[0][0]
if closed != []:
closed = closed[0][0]
if deadline != []:
deadline = deadline[0][0]
if scheduled or deadline or closed:
self._append(current,self.Element(scheduled, deadline,closed))
else:
self.treated = False
return current
class Element(OrgElement):
"""Schedule is an element taking into account DEADLINE, SCHEDULED and CLOSED parameters of elements"""
DEADLINE = 1
SCHEDULED = 2
CLOSED = 4
TYPE = "SCHEDULE_ELEMENT"
def __init__(self,scheduled=[],deadline=[],closed=[]):
OrgElement.__init__(self)
self.type = 0
if scheduled != []:
self.type = self.type | self.SCHEDULED
self.scheduled = OrgDate(scheduled)
if deadline != []:
self.type = self.type | self.DEADLINE
self.deadline = OrgDate(deadline)
if closed != []:
self.type = self.type | self.CLOSED
self.closed = OrgDate(closed)
def _output(self):
"""Outputs the Schedule element in text format (e.g SCHEDULED: <2010-10-10 10:10>)"""
output = ""
if self.type & self.SCHEDULED:
output = output + "SCHEDULED: "+self.scheduled.get_value()+" "
if self.type & self.DEADLINE:
output = output + "DEADLINE: "+self.deadline.get_value()+" "
if self.type & self.CLOSED:
output = output + "CLOSED: "+self.closed.get_value()+" "
if output != "":
output = output.rstrip() + "\n"
return output
class OrgDrawer(OrgPlugin):
"""A Plugin for drawers"""
def __init__(self):
OrgPlugin.__init__(self)
self.regexp = re.compile("^(?:\s*?)(?::)(\S.*?)(?::)\s*(.*?)$")
def _treat(self,current,line):
drawer = self.regexp.search(line)
if isinstance(current, OrgDrawer.Element): # We are in a drawer
if drawer:
if drawer.group(1) == "END": # Ending drawer
current = current.parent
elif drawer.group(2): # Adding a property
self._append(current,self.Property(drawer.group(1),drawer.group(2)))
else: # Adding text in drawer
self._append(current,line.rstrip("\n"))
elif drawer: # Creating a drawer
current = self._append(current,OrgDrawer.Element(drawer.group(1)))
else:
self.treated = False
return current
return current # It is a drawer, change the current also (even if not modified)
class Element(OrgElement):
"""A Drawer object, containing properties and text"""
TYPE = "DRAWER_ELEMENT"
def __init__(self,name=""):
OrgElement.__init__(self)
self.name = name
def _output(self):
output = ":" + self.name + ":\n"
for element in self.content:
output = output + str(element) + "\n"
output = output + self.indent + ":END:\n"
return output
class Property(OrgElement):
"""A Property object, used in drawers."""
def __init__(self,name="",value=""):
OrgElement.__init__(self)
self.name = name
self.value = value
def _output(self):
"""Outputs the property in text format (e.g. :name: value)"""
return ":" + self.name + ": " + self.value--- Class OrgTable
- [ ] Table edition (must add separators, cell length, length calculator…)
class OrgTable(OrgPlugin):
"""A plugin for table managment"""
def __init__(self):
OrgPlugin.__init__(self)
self.regexp = re.compile("^\s*\|")
def _treat(self,current,line):
table = self.regexp.match(line)
if table:
if not isinstance(current,self.Element):
current = current.append(self.Element())
current.append(line.rstrip().strip("|").split("|"))
else:
if isinstance(current,self.Element):
current = current.parent
self.treated = False
return current
class Element(OrgElement):
"""
A Table object
"""
TYPE = "TABLE_ELEMENT"
def __init__(self):
OrgElement.__init__(self)
def _output(self):
output = ""
for element in self.content:
output = output + "|"
for cell in element:
output = output + str(cell) + "|"
output = output + "\n"
return output
--- Class OrgNode
- [ ] Add the intra-header scheduling
class OrgNode(OrgPlugin):
def __init__(self):
OrgPlugin.__init__(self)
self.regexp = re.compile("^(\*+)\s*(\[.*\])?\s*(.*)$")
self.keepindent = False # If the line starts by an indent, it is not a node
def _treat(self,current,line):
heading = self.regexp.findall(line)
if heading: # We have a heading
if current.parent :
current.parent.append(current)
# Is that a new level ?
if (len(heading[0][0]) > current.level): # Yes
parent = current # Parent is now the current node
else:
parent = current.parent # If not, the parent of the current node is the parent
# If we are going back one or more levels, walk through parents
while len(heading[0][0]) < current.level:
current = current.parent
parent = current.parent
# Creating a new node and assigning parameters
current = OrgNode.Element()
current.level = len(heading[0][0])
current.heading = re.sub(":([\w]+):","",heading[0][2]) # Remove tags
current.priority = heading[0][1]
current.parent = parent
# Looking for tags
heading_without_links = re.sub(" \[(.+)\]","",heading[0][2])
current.tags = re.findall(":([\w]+):",heading_without_links)
else:
self.treated = False
return current
def _close(self,current):
# Add the last node
if current.level>0:
current.parent.append(current)
class Element(OrgElement):
# Defines an OrgMode Node in a structure
# The ID is auto-generated using uuid.
# The level 0 is the document itself
TYPE = "NODE_ELEMENT"
def __init__(self):
OrgElement.__init__(self)
self.content = []
self.level = 0
self.heading = ""
self.priority = ""
self.tags = []
# TODO Scheduling structure
def _output(self):
output = ""
if hasattr(self,"level"):
output = output + "*"*self.level
if self.parent is not None:
output = output + " "
if self.priority:
output = output + self.priority + " "
output = output + self.heading
for tag in self.tags:
output= output + ":" + tag + ":"
output = output + "\n"
for element in self.content:
output = output + element.__str__()
return output
def append_clean(self,element):
if isinstance(element,list):
self.content.extend(element)
else:
self.content.append(element)
self.reparent_cleanlevels(self)
def reparent_cleanlevels(self,element=None,level=None):
"""
Reparent the childs elements of 'element' and make levels simpler.
Useful after moving one tree to another place or another file.
"""
if element == None:
element = self.root
if hasattr(element,"level"):
if level == None:
level = element.level
else:
element.level = level
if hasattr(element,"content"):
for child in element.content:
if hasattr(child,"parent"):
child.parent = element
self.reparent_cleanlevels(child,level+1) class OrgDataStructure(OrgElement):
"""
Data structure containing all the nodes
The root property contains a reference to the level 0 node
"""
root = None
TYPE = "DATASTRUCTURE_ELEMENT"
def __init__(self):
OrgElement.__init__(self)
self.plugins = []
self.load_plugins(OrgTable(),OrgDrawer(),OrgNode(),OrgSchedule(),OrgClock())
# Add a root element
# The root node is a special node (no parent) used as a container for the file
self.root = OrgNode.Element()
self.root.parent = None
self.level = 0
def load_plugins(self,*arguments,**keywords):
"""
Used to load plugins inside this DataStructure
"""
for plugin in arguments:
self.plugins.append(plugin)
def load_from_file(self,name):
"""
Used to load an org-file inside this DataStructure
"""
current = self.root
file = open(name,'r')
for line in file:
for plugin in self.plugins:
current = plugin.treat(current,line)
if plugin.treated: # Plugin found something
treated = True
break;
else:
treated = False
if not treated and line is not None: # Nothing special, just content
current.append(line)
for plugin in self.plugins:
current = plugin.close(current)
file.close()
def save_to_file(self,name,node=None):
"""
Used to save an org-file corresponding to this DataStructure
"""
output = open(name,'w')
if node == None:
node = self.root
output.write(str(node))
output.close()import PyOrgMode
import tempfile
import unittest
class TestParser(unittest.TestCase):
"""Test the org file parser with a simple org structure"""
def setUp(self):
"""Parse the org structure from a temporary file"""
orgfile = tempfile.NamedTemporaryFile()
orgfile.write('\n'.join((
'* one',
'* two',
'** two point one',
'* three',
'')).encode('UTF-8'))
orgfile.flush()
self.tree = PyOrgMode.OrgDataStructure()
try:
self.tree.load_from_file(orgfile.name)
finally:
orgfile.close()
def test_has_three_top_level_headings(self):
"""The example has three top-level headings"""
self.assertEqual(len(self.tree.root.content), 3)
def test_second_item_has_a_subheading(self):
"""The second top-level heading has one subheading"""
self.assertEqual(len(self.tree.root.content[1].content), 1)
if __name__ == '__main__':
unittest.main()"""Tests for parsing and outputting a simple .org test file
You need the fr_FR.UTF-8 locale to run these tests
"""
import locale
import PyOrgMode
try:
import unittest2 as unittest
except ImportError:
import unittest
def _normalize_ignored(line):
"""Normalize a line to ignore differences which aren't yet handled"""
line = line.replace(':ORDERED: t', ':ORDERED: t')
return line
class TestExampleOrgFile(unittest.TestCase):
def test_test_org(self):
test = PyOrgMode.OrgDataStructure()
test.load_from_file("test.org")
locale.setlocale(locale.LC_ALL, 'fr_FR.UTF-8')
test.save_to_file("output.org")
original = [_normalize_ignored(line) for line in open("test.org")]
saved = [_normalize_ignored(line) for line in open("output.org")]
self.assertEqual(saved, original)
if __name__ == '__main__':
unittest.main()import PyOrgMode
import copy
try:
import unittest2 as unittest
except ImportError:
import unittest
def Get_Scheduled_Elements(element, data=[]):
"""
Grab the data from all scheduled elements for all the tree defined by 'element' recursively.
Returns all the elements as an array.
"""
if hasattr(element,"content"):
for child in element.content:
if hasattr(child,"TYPE"):
if child.TYPE == "SCHEDULE_ELEMENT":
# This element is scheduled, we are creating a copy of it
data.append(copy.deepcopy(child.parent))
Get_Scheduled_Elements(child,data)
return data
class TestAgenda(unittest.TestCase):
def test_agenda(self):
# Creating the input and output files data structures
input_file = PyOrgMode.OrgDataStructure()
output_file = PyOrgMode.OrgDataStructure()
# Loading from agenda.org file
input_file.load_from_file("agenda.org")
# Get the scheduled elements (those with SCHEDULE, DEADLINE in them, not in the node name)
scheduled_elements = Get_Scheduled_Elements(input_file.root)
# Assign these element to the root (reparent each elements recursively, relevel them cleanly)
output_file.root.append_clean(scheduled_elements)
output_file.save_to_file("test_scheduled_output.org")
saved = open("test_scheduled_output.org").readlines()
self.assertEqual(saved, ['* Element 1\n',
' SCHEDULED: <2011-02-08>\n',
'* Element 3\n',
' DEADLINE: <2011-02-08>\n',
'** Test\n',
'** Element 4\n',
' SCHEDULED: <2011-02-08>\n',
'*** Couic\n',
'* Element 4\n',
' SCHEDULED: <2011-02-08>\n',
'** Couic\n'])
if __name__ == '__main__':
unittest.main()This part is used for internal testing. It allows you to test some piece of code inside your org document.
Just use C-c C-c to execute this codeimport re
test_regexp = re.compile("")
result = test_regexp.findall("")
return(result)